Covered Information Disentanglement: Model Transparency via Unbiased Permutation Importance

Model transparency is a prerequisite in many domains and an increasingly popular area machine learning research. In the medical domain, for instance, unveiling mechanisms behind disease often has higher priority than diagnostic itself since it might dictate or guide potential treatments research directions. One of most approaches to explain model global predictions permutation importance where performance on permuted data benchmarked against baseline. However, this method other related will undervalue feature presence covariates these cover part its provided information. To address issue, we propose Covered Information Disentanglement CID, framework that considers all information overlap correct values by importance. We further show how compute CID efficiently when coupled with Markov random fields. demonstrate efficacy adjusting first controlled toy dataset discuss effect real-world data.